Display system of work machine and method of controlling the same

ABSTRACT

A display system of a work machine includes an obtaining unit that obtains motion information and operation information of the work machine, a motion image generator that generates a motion image of the work machine and an operation image corresponding to the motion image of the work machine based on the motion information and the operation information obtained by the obtaining unit, and a display that shows the motion image and the operation image of the work machine.

TECHNICAL FIELD

The present invention relates to a work machine.

BACKGROUND ART

A work machine such as a wheel loader includes a bucket pivotable in adirection of dumping at a tip end of a boom pivotable in anupward/downward direction. An operator performs an excavation work byoperating an operation apparatus to pivot the bucket in the direction ofdumping to set the bucket at a substantially horizontal position and tothereafter run the work machine to push the bucket into a pile of soil.An object is thus loaded into the bucket. The operator revolves the boomor a vehicular body to have the work machine face a transportationmachine such as a dump truck, and raises the boom above a box. As theoperator pivots the bucket in the direction of dumping, the objectloaded in the bucket falls on the box and the object is transferred tothe transportation machine. By repeating such a cycle a plurality oftimes, a loading work is performed.

In a motion of a work machine such as a wheel loader, an accelerator forrunning the work machine should be operated and levers for operating theboom and the bucket should be operated to control motions of the bucket.Therefore, it is not easy to realize efficient motions and skills arerequired. Therefore, a function to check a motion state of the workmachine to allow training for driving is demanded.

In this connection, for example, Japanese Patent Laying-Open No.2016-89388 discloses a technique to transmit information to a remotefacility to give a support image for supporting a work by an operator atthe remote facility. An image in consideration of relative positionalrelation between the work machine and a transportation vehicle isgenerated as the support image. In the publication, however, a supportimage relating to an attitude of the work machine during works is merelygiven, and on which operation command the attitude of the work machineis based cannot readily be recognized.

CITATION LIST Patent Literature

PTL 1: Japanese Patent Laying-Open No. 2016-89388

SUMMARY OF INVENTION Technical Problem

In this connection, the present invention was made to solve the problemabove, and an object thereof is to provide a display system of a workmachine that allows easy recognition of not only a motion state but alsoan operation state of the work machine and a method of controlling thesame.

Solution to Problem

A display system of a work machine according to the present inventionincludes an obtaining unit that obtains motion information and operationinformation of the work machine, a motion image generator that generatesa motion image of the work machine and an operation image correspondingto the motion image of the work machine based on the motion informationand the operation information obtained by the obtaining unit, and adisplay that shows the motion image and the operation image of the workmachine.

A method of controlling a display system of a work machine according tothe present invention includes obtaining motion information andoperation information of the work machine, generating a motion image ofthe work machine and an operation image corresponding to the motionimage of the work machine based on the obtained motion information andoperation information, and showing the motion image and the operationimage of the work machine.

Advantageous Effects of Invention

The display system of the work machine and the method of controlling thesame according to the present invention can easily recognize not only amotion state but also an operation state of the work machine.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a side view of a wheel loader 1 as an exemplary work machineaccording to an embodiment.

FIG. 2 is a schematic block diagram showing a configuration of theentire system including wheel loader 1 according to the embodiment.

FIG. 3 is a schematic diagram illustrating a work step of wheel loader 1based on the embodiment.

FIG. 4 shows a table showing a method of distinguishing a work step ofwheel loader 1 based on the embodiment.

FIG. 5 is a diagram illustrating a functional block of a secondprocessor 70 according to the embodiment.

FIG. 6 is a diagram illustrating a work machine table stored in a memory73 according to the embodiment.

FIG. 7 is a flowchart illustrating event registration processing bysecond processor 70 according to the embodiment.

FIG. 8 is a diagram illustrating a detailed functional block of a motionimage generator 82 according to the embodiment.

FIG. 9 is a diagram illustrating a work screen 200 on a display 72according to the embodiment.

FIG. 10 is a flowchart illustrating replay position selection processingby second processor 70 according to the embodiment.

DESCRIPTION OF EMBODIMENTS

Though an embodiment according to the present invention will bedescribed below with reference to the drawings, the present invention isnot limited thereto. Constituent elements in each embodiment describedbelow can be combined as appropriate. Some of the constituent elementsmay not be used.

[Overall Configuration]

A wheel loader 1 as an exemplary work machine will be described in anembodiment. FIG. 1 is a side view of wheel loader 1 as an exemplary workmachine according to the embodiment. As shown in FIG. 1, wheel loader 1includes a vehicular body frame 2, a work implement 3, a traveling unit4, and a cab 5. A vehicular body of wheel loader 1 is constituted ofvehicular body frame 2, cab 5, and the like. Work implement 3 andtraveling unit 4 are attached to the vehicular body of wheel loader 1.

Traveling unit 4 runs the vehicular body of wheel loader 1 and includesrunning wheels 4 a and 4 b. Wheel loader 1 can be self-propelled asrunning wheels 4 a and 4 b are rotationally driven, and can perform adesired work with work implement 3.

Vehicular body frame 2 includes a front frame 11 and a rear frame 12.Front frame 11 and rear frame 12 are attached to each other in a mannerswingable in a lateral direction. A steering cylinder 13 is attached tofront frame 11 and rear frame 12.

Steering cylinder 13 is a hydraulic cylinder. As steering cylinder 13extends and contracts as being driven by hydraulic oil from a steeringpump (not shown), a direction of travel of wheel loader 1 is laterallychanged.

A direction in which wheel loader 1 travels in straight lines is hereinreferred to as a fore/aft direction of wheel loader 1. In the fore/aftdirection of wheel loader 1, a side where work implement 3 is arrangedwith respect to vehicular body frame 2 is defined as the fore directionand a direction opposite to the fore direction is defined as the aftdirection. A lateral direction of wheel loader 1 is a directionorthogonal to the fore/aft direction in a plan view. A right side and aleft side in the lateral direction in facing front are defined as aright direction and a left direction, respectively. An upward/downwarddirection of wheel loader 1 is a direction orthogonal to the planedefined by the fore/aft direction and the lateral direction. A side inthe upward/downward direction where the ground is located is defined asa lower side and a side where the sky is located is defined as an upperside.

The fore/aft direction refers to a fore/aft direction of an operator whosits at an operator's seat in cab 5. The lateral direction refers to alateral direction of the operator who sits at the operator's seat. Thelateral direction refers to a direction of a vehicle width of wheelloader 1. The upward/downward direction refers to an upward/downwarddirection of the operator who sits at the operator's seat. A directionin which the operator sitting at the operator's seat faces is defined asthe fore direction and a direction behind the operator sitting at theoperator's seat is defined as the aft direction. A right side and a leftside at the time when the operator sitting at the operator's seat facesfront are defined as the right direction and the left direction,respectively. A foot side of the operator who sits at the operator'sseat is defined as a lower side, and a head side is defined as an upperside.

Work implement 3 and running wheel (front wheel) 4 a are attached tofront frame 11. Work implement 3 includes a boom 14 and a bucket 6. Abase end of boom 14 is rotatably attached to front frame 11 by a boompin 10. Bucket 6 is rotatably attached to boom 14 by a bucket pin 17located at a tip end of boom 14. Front frame 11 and boom 14 are coupledto each other by a boom cylinder 16. Boom cylinder 16 is a hydrauliccylinder. As boom cylinder 16 extends and contracts as being driven byhydraulic oil from a work implement pump 25 (see FIG. 2), boom 14 movesupward and downward. Boom cylinder 16 drives boom 14.

Work implement 3 further includes a bell crank 18, a tilt cylinder 19,and a tilt rod 15. Bell crank 18 is rotatably supported on boom 14 by asupport pin 18 a located substantially in the center of boom 14. Tiltcylinder 19 couples a base end of bell crank 18 and front frame 11 toeach other. Tilt rod 15 couples a tip end of bell crank 18 and bucket 6to each other. Tilt cylinder 19 is a hydraulic cylinder. As tiltcylinder 19 extends and contracts as being driven by hydraulic oil fromwork implement pump 25 (see FIG. 2), bucket 6 pivots upward anddownward. Tilt cylinder 19 drives bucket 6.

Cab 5 and running wheel (rear wheel) 4 b are attached to rear frame 12.Cab 5 is arranged in the rear of boom 14. Cab 5 is carried on vehicularbody frame 2. A seat where an operator sits and an operation apparatusare arranged in cab 5.

A position detection sensor 64 is arranged on an upper celling side ofcab 5. Position detection sensor 64 includes a GNSS antenna and a globalcoordinate operator. The GNSS antenna is an antenna for a real timekinematic-global navigation satellite system (RTK-GNSS). An inertialmeasurement unit (IMU) 66 is arranged in cab 5. IMU 66 detects aninclination of vehicular body frame 2. IMU 66 detects an angle ofinclination of vehicular body frame 2 with respect to the fore/aftdirection and the lateral direction.

FIG. 2 is a schematic block diagram showing a configuration of theentire system including wheel loader 1 according to the embodiment.Referring to FIG. 2, the entire system according to the embodimentincludes wheel loader 1 and a second processor provided to be able toestablish wireless or wired communication with wheel loader 1.

Wheel loader 1 includes an engine 20, a motive power extraction unit 22,a motive power transmission mechanism 23, a cylinder driving unit 24, afirst angle detector 29, a second angle detector 48, a pivot mechanism60, and a first processor 30 (a controller).

Engine 20 is, for example, a diesel engine. Output from engine 20 iscontrolled by adjusting an amount of fuel to be injected into a cylinderof engine 20. Engine 20 is provided with a temperature sensor 31.Temperature sensor 31 outputs a detection signal representing atemperature to first processor 30.

Motive power extraction unit 22 is an apparatus that distributes outputfrom engine 20 to motive power transmission mechanism 23 and cylinderdriving unit 24. Motive power transmission mechanism 23 is a mechanismthat transmits driving force from engine 20 to front wheel 4 a and rearwheel 4 b, and it is implemented, for example, by a transmission. Motivepower transmission mechanism 23 changes a speed of rotation of an inputshaft 21 and outputs resultant rotation to an output shaft 23 a. Avehicle speed detection unit 27 that detects a speed of wheel loader 1is attached to output shaft 23 a of motive power transmission mechanism23. Wheel loader 1 includes vehicle speed detection unit 27.

Vehicle speed detection unit 27 is implemented, for example, by avehicle speed sensor. Vehicle speed detection unit 27 detects a speed ofmovement of wheel loader 1 by traveling unit 4 (FIG. 1) by detecting arotation speed of output shaft 23 a. Vehicle speed detection unit 27functions as a rotation sensor that detects a rotation speed of outputshaft 23 a. Vehicle speed detection unit 27 functions as a movementdetector that detects movement by traveling unit 4. Vehicle speeddetection unit 27 outputs a detection signal representing a vehiclespeed of wheel loader 1 to first processor 30.

Cylinder driving unit 24 includes work implement pump 25 and a controlvalve 26. Output from engine 20 is transmitted to work implement pump 25through motive power extraction unit 22. Hydraulic oil delivered fromwork implement pump 25 is supplied to boom cylinder 16 and tilt cylinder19 through control valve 26.

First hydraulic pressure detectors 28 a and 28 b that detect a hydraulicpressure in an oil chamber in boom cylinder 16 are attached to boomcylinder 16. Wheel loader 1 includes first hydraulic pressure detectors28 a and 28 b. First hydraulic pressure detectors 28 a and 28 b include,for example, a pressure sensor 28 a for head pressure detection and apressure sensor 28 b for bottom pressure detection.

Pressure sensor 28 a is attached to a head side of boom cylinder 16.Pressure sensor 28 a can detect a pressure (a head pressure) ofhydraulic oil in the oil chamber on a side of a cylinder head of boomcylinder 16. Pressure sensor 28 a outputs a detection signalrepresenting a head pressure of boom cylinder 16 to first processor 30.Pressure sensor 28 b is attached to a bottom side of boom cylinder 16.Pressure sensor 28 b can detect a pressure (a bottom pressure) ofhydraulic oil in the oil chamber on a side of a cylinder bottom of boomcylinder 16. Pressure sensor 28 b outputs a detection signalrepresenting a bottom pressure of boom cylinder 16 to first processor30.

For example, a potentiometer attached to boom pin 10 is employed asfirst angle detector 29. First angle detector 29 detects a boom anglerepresenting a lift angle (a tilt angle) of boom 14. First angledetector 29 outputs a detection signal representing a boom angle tofirst processor 30. Specifically, as shown in FIG. 1, a boom angle θrepresents an angle of a straight line LB extending in a direction fromthe center of boom pin 10 toward the center of bucket pin 17 withrespect to a horizontal line extending forward from the center of boompin 10. A case that straight line LB is horizontal is defined as boomangle θ=0°. A case that straight line LB is located above the horizontalline is defined as a positive boom angle θ. A case that straight line LBis located below the horizontal line is defined as a negative boom angleθ. A stroke sensor arranged in boom cylinder 16 may be employed as firstangle detector 29.

For example, a potentiometer attached to support pin 18 a is employed assecond angle detector 48. Second angle detector 48 detects a bucketangle representing a tilt angle of bucket 6 with respect to boom 14 bydetecting an angle of bell crank 18 (bell crank angle) with respect toboom 14. Second angle detector 48 outputs a detection signalrepresenting a bucket angle to first processor 30. The bucket angle is,for example, an angle formed between straight line LB and a straightline that connects the center of bucket pin 17 and a cutting edge 6 a ofbucket 6 to each other. A stroke sensor arranged in tilt cylinder 19 maybe employed as second angle detector 48.

Pivot mechanism 60 pivotably couples front frame 11 and rear frame 12 toeach other. Front frame 11 is pivoted with respect to rear frame 12 byextending and contracting an articulation cylinder coupled between frontframe 11 and rear frame 12. By angling (articulating) front frame 11with respect to rear frame 12, a radius of revolution in revolution ofthe wheel loader can be made smaller and a ditch digging work or agrading work by offset running can be done. Pivot mechanism 60 isprovided with an articulation angle sensor 61. Articulation angle sensor61 detects an articulation angle. Articulation angle sensor 61 outputs adetection signal representing the articulation angle to first processor30.

Position detection sensor 64 outputs a detection signal representing aposition of wheel loader 1 to first processor 30. IMU 66 outputs adetection signal representing an angle of inclination of wheel loader 1to first processor 30.

As shown in FIG. 2, wheel loader 1 includes in cab 5, an operationapparatus operated by an operator. The operation apparatus includes aforward and rearward travel switching apparatus 49, an acceleratoroperation apparatus 51, a boom operation apparatus 52, a shift changeoperation apparatus 53, a bucket operation apparatus 54, and a brakeoperation apparatus 58.

Forward and rearward travel switching apparatus 49 includes a forwardand rearward travel switching operation member 49 a and a forward andrearward travel switching detection sensor 49 b. Forward and rearwardtravel switching operation member 49 a is operated by an operator forindicating switching between forward travel and rearward travel of thevehicle. Forward and rearward travel switching operation member 49 a canbe switched to a position of each of forward travel (F), neutral (N),and rearward travel (R). Forward and rearward travel switching detectionsensor 49 b detects a position of forward and rearward travel switchingoperation member 49 a. Forward and rearward travel switching detectionsensor 49 b outputs to first processor 30, a detection signal (forwardtravel, neutral, or rearward travel) representing a command to travelforward or rearward indicated by a position of forward and rearwardtravel switching operation member 49 a. Forward and rearward travelswitching apparatus 49 includes an FNR switch lever capable of switchingamong forward travel (F), neutral (N), and rearward travel (R).

Accelerator operation apparatus 51 includes an accelerator operationmember 51 a and an accelerator operation detection unit 51 b.Accelerator operation member 51 a is operated by an operator for settinga target rotation speed of engine 20. Accelerator operation detectionunit 51 b detects an amount of operation onto accelerator operationmember 51 a (an amount of accelerator operation). Accelerator operationdetection unit 51 b outputs a detection signal representing an amount ofaccelerator operation to first processor 30.

Brake operation apparatus 58 includes a brake operation member 58 a anda brake operation detection unit 58 b. Brake operation member 58 a isoperated by an operator for controlling deceleration force of wheelloader 1. Brake operation detection unit 58 b detects an amount ofoperation onto brake operation member 58 a (an amount of brakeoperation). Brake operation detection unit 58 b outputs a detectionsignal representing an amount of brake operation to first processor 30.A pressure of brake oil may be used as an amount of brake operation.

Boom operation apparatus 52 includes a boom operation member 52 a and aboom operation detection unit 52 b. Boom operation member 52 a isoperated by an operator for raising or lowering boom 14. Boom operationdetection unit 52 b detects a position of boom operation member 52 a.Boom operation detection unit 52 b outputs to first processor 30, adetection signal representing a command to raise or lower boom 14indicated by the position of boom operation member 52 a.

Shift change operation apparatus 53 includes a shift change operationmember 53 a and a shift change operation detection unit 53 b. Shiftchange operation member 53 a is operated by an operator for controllingshift change from input shaft 21 to output shaft 23 a in motive powertransmission mechanism 23. Shift change operation detection unit 53 bdetects a position of shift change operation member 53 a. Shift changeoperation detection unit 53 b outputs a shift change detection commandindicated by the position of shift change operation member 53 a to firstprocessor 30.

Bucket operation apparatus 54 includes a bucket operation member 54 aand a bucket operation detection unit 54 b. Bucket operation member 54 ais operated by an operator for causing bucket 6 to carry out anexcavation motion or a dumping motion. Bucket operation detection unit54 b detects a position of bucket operation member 54 a. Bucketoperation detection unit 54 b outputs to first processor 30, a detectionsignal representing a command for an operation in a tilt-back directionor a dump direction of bucket 6 indicated by a position of bucketoperation member 54 a.

Articulation operation apparatus 55 includes an articulation operationmember 55 a and an articulation operation detection unit 55 b.Articulation operation member 55 a is operated by an operator forangling (articulating) front frame 11 with respect to rear frame 12 withpivot mechanism 60 being interposed. Articulation operation detectionunit 55 b detects a position of articulation operation member 55 a.Articulation operation detection unit 55 b outputs to first processor30, a detection signal representing a left angling command or a rightangling command indicated by a position of articulation operation member55 a.

First processor 30 is implemented by a microcomputer including a storagesuch as a random access memory (RAM) or a read only memory (ROM) and acomputing device such as a central processing unit (CPU). Firstprocessor 30 may be implemented as some of functions of a controller ofwheel loader 1 that controls motions of engine 20, work implement 3(boom cylinder 16, tilt cylinder 19, and the like), and motive powertransmission mechanism 23. A signal representing a forward and rearwardtravel command detected by forward and rearward travel switchingapparatus 49, a signal representing a vehicle speed of wheel loader 1detected by vehicle speed detection unit 27, a signal representing aboom angle detected by first angle detector 29, a signal representing ahead pressure of boom cylinder 16 detected by pressure sensor 28 a, anda signal representing a bottom pressure of boom cylinder 16 detected bypressure sensor 28 b are mainly input to first processor 30.

Wheel loader 1 further includes a display 40 and an output unit 45.Display 40 is implemented by a monitor arranged in cab 5 and viewed byan operator.

Output unit 45 outputs work machine motion information including motioninformation of wheel loader 1 to a server (a second processor 70)provided outside wheel loader 1. Output unit 45 may output work machinemotion information including motion information of wheel loader 1 everyprescribed period or may collectively output work machine motioninformation over a plurality of periods. Output unit 45 may have acommunication function such as wireless communication and maycommunicate with second processor 70. Alternatively, output unit 45 maybe implemented, for example, by an interface of a portable storage (suchas a memory card) that can be accessed from second processor 70. Secondprocessor 70 includes a display that performs a monitor function and canshow a motion image based on work machine motion information output fromoutput unit 45. Second processor 70 is provided at a position differentfrom a position where wheel loader 1 is provided, and a motion imageduring works by wheel loader 1 can be recognized on a display at aremote location by way of example.

[Work Step of Wheel Loader 1 and Distinction Thereof]

Wheel loader 1 in the present embodiment performs an excavation motionfor scooping an excavated object such as soil in bucket 6 and a loadingmotion for loading objects (an excavated object 100) in bucket 6 onto atransportation machine such as a dump truck 110.

FIG. 3 is a schematic diagram illustrating a work step of wheel loader 1based on the embodiment. Wheel loader 1 excavates excavated object 100and loads excavated object 100 on a transportation machine such as dumptruck 110 by successively repeating a plurality of steps as follows.

As shown in FIG. 3 (A), wheel loader 1 travels forward toward excavatedobject 100. In this unloaded forward travel step, an operator operatesboom cylinder 16 and tilt cylinder 19 to set work implement 3 to anexcavation attitude in which the tip end of boom 14 is located at a lowposition and bucket 6 is horizontally oriented, and moves wheel loader 1forward toward excavated object 100.

As shown in FIG. 3 (B), the operator moves wheel loader 1 forward untilcutting edge 6 a of bucket 6 is pushed into excavated object 100. Inthis excavation (pushing) step, cutting edge 6 a of bucket 6 is pushedinto excavated object 100.

As shown in FIG. 3 (C), the operator thereafter operates boom cylinder16 to raise bucket 6 and operates tilt cylinder 19 to tilt back bucket6. In this excavation (scooping) step, bucket 6 is raised along a buckettrack L as shown with a curved arrow in the figure and excavated object100 is scooped into bucket 6. An excavation work for scooping excavatedobject 100 is thus performed.

Depending on a type of excavated object 100, the scooping step may becompleted simply by tilting back bucket 6 once. Alternatively, in thescooping step, a motion to tilt back bucket 6, set the bucket to aneutral position, and tilt back the bucket again may be repeated.

As shown in FIG. 3 (D), after excavated object 100 is scooped intobucket 6, the operator moves wheel loader 1 rearward in a loadedrearward travel step. The operator may raise the boom while moving thevehicle rearward, or may raise the boom while moving the vehicle forwardin FIG. 3 (E).

As shown in FIG. 3 (E), the operator moves wheel loader 1 forward to becloser to dump truck 110 while keeping bucket 6 raised or raising bucket6. As a result of this loaded forward travel step, bucket 6 is locatedsubstantially directly above a box of dump truck 110.

As shown in FIG. 3 (F), the operator dumps the excavated object frombucket 6 at a prescribed position and loads objects (excavated object)in bucket 6 on the box of dump truck 110. This step is what is called asoil ejection step. Thereafter, the operator lowers boom 14 and returnsbucket 6 to the excavation attitude while the operator moves wheelloader 1 rearward. The above is typical steps defining one cycle of theexcavation and loading work.

FIG. 4 shows a table showing a method of distinguishing a work step ofwheel loader 1 based on the embodiment. In the table shown in FIG. 4, arow of “work step” at the top lists names of work steps shown in FIG. 3(A) to (F). In rows of “forward and rearward travel switching lever,”“operation of work implement,” and “pressure of cylinder of workimplement” below, various criteria used by first processor 30 (FIGS. 2and 3) for determining under which step a current work step falls areshown. More specifically, in the row of “forward and rearward travelswitching lever,” criteria for a forward and rearward travel switchinglever are shown with a circle.

In the row of “operation of work implement,” criteria for an operationby an operator onto work implement 3 are shown with a circle. Morespecifically, in a row of “boom”, criteria for an operation onto boom 14are shown, and in a row of “bucket”, criteria for an operation ontobucket 6 are shown.

In the row of “pressure of cylinder of work implement,” criteria for acurrent hydraulic pressure of the cylinder of work implement 3 such as ahydraulic pressure of a cylinder bottom chamber of boom cylinder 16 areshown. Four reference values A, B, C, and P are set in advance for ahydraulic pressure, a plurality of pressure ranges (a range lower thanreference value P, a range of reference values A to C, a range ofreference values B to P, and a range lower than reference value C) aredefined by reference values A, B, C, and P, and these pressure rangesare set as the criteria. Magnitude of four reference values A, B, C, andP is defined as A>B>C>P.

By using a combination of criteria for “forward and rearward travelswitching lever,” “boom”, “bucket”, and “pressure of cylinder of workimplement” for each work step as above, first processor 30 candistinguish a currently performed step.

A specific operation of first processor 30 when control shown in FIG. 4is carried out will be described below. A combination of criteria for“forward and rearward travel switching lever,” “boom”, “bucket”, and“pressure of cylinder of work implement” corresponding to each work stepshown in FIG. 4 is stored in advance in a storage 30 j (FIG. 2). Firstprocessor 30 recognizes a currently selected forward and rearward travelswitching lever (F, N, or R) based on a signal from forward and rearwardtravel switching apparatus 49. First processor 30 recognizes a type of acurrent operation onto boom 14 (lowering, neutral, or raising) based ona signal from boom operation detection unit 52 b. First processor 30recognizes a type of a current operation onto bucket 6 (dump, neutral,or tilt back) based on a signal from bucket operation detection unit 54b. First processor 30 recognizes a current hydraulic pressure of thecylinder bottom chamber of boom cylinder 16 based on a signal frompressure sensor 28 b shown in FIG. 2.

First processor 30 compares combination of the recognized forward andrearward travel switching lever, the type of the operation onto theboom, the type of the operation onto the bucket, and the hydraulicpressure of the lift cylinder at the current time point (that is, acurrent state of work) with combination of criteria for “forward andrearward travel switching lever,” “boom”, “bucket”, and “pressure ofcylinder of work implement” corresponding to each work step stored inadvance. As a result of this comparison processing, first processor 30determines to which work step the combination of criteria which matchesbest with the current state of work corresponds. The combination ofcriteria corresponding to the excavation and loading motion shown inFIG. 4 is as follows by way of example.

In the unloaded forward travel step, the forward and rearward travelswitching lever is set to F, the operation of the boom and the operationof the bucket are both set to neutral, and the pressure of the cylinderof the work implement is lower than reference value P. In the excavation(pushing) step, the forward and rearward travel switching lever is setto F, the operation of the boom and the operation of the bucket are bothneutral, and the pressure of the cylinder of the work implement iswithin the range of reference values A to C. In the excavation(scooping) step, the forward and rearward travel switching lever is setto F or R, the operation of the boom is raising or neutral, theoperation of the bucket is tilt back, and the pressure of the cylinderof the work implement is within the range of reference values A to C.For an operation of the bucket, such a criterion that tilt back andneutral are alternately repeated may further be added because, dependingon a state of an excavated object, a motion to tilt back bucket 6, setthe bucket to a neutral position, and tilt back the bucket again may berepeated. In the loaded rearward travel step, the forward and rearwardtravel switching lever is set to R, the operation of the boom is neutralor raising, the operation of the bucket is neutral, and the pressure ofthe cylinder of the work implement is within the range of referencevalues B to P. In the loaded forward travel step, the forward andrearward travel switching lever is set to F, the operation of the boomis raising or neutral, the operation of the bucket is neutral, and thepressure of the cylinder of the work implement is within the range ofreference values B to P. In the soil ejection step, the forward andrearward travel switching lever is set to F, the operation of the boomis raising or neutral, the operation of the bucket is dump, and thepressure of the cylinder of the work implement is within the range ofreference values B to P. In the rearward travel⋅boom lowering step, theforward and rearward travel switching lever is set to R, the operationof the boom is lowering, the operation of the bucket is tilt back, andthe pressure of the cylinder of the work implement is lower thanreference value P.

Information on the work step determined by first processor 30 is outputas a part of work machine motion information to second processor 70through output unit 45. Though a scheme for determination of a work stepby first processor 30 is described in the present example, the work stepmay be determined by second processor 70 without particularly beinglimited as such.

[Functional Configuration of Second Processor 70]

FIG. 5 is a diagram illustrating a functional block of second processor70 according to the embodiment. Referring to FIG. 5, second processor 70includes an input unit 71, a display 72, a memory 73, a communicationunit 74, and a CPU 75.

Input unit 71 includes a mouse, a keyboard, a controller, a touch panel,and the like. An input command is generated by operating input unit 71.For example, an input command is generated by operating a mouse,operating a keyboard, operating a button on the controller, orperforming a touching operation onto the touch panel.

Display 72 includes a display of liquid crystals or the like. Memory 73includes a storage such as a RAM or a ROM. Memory 73 stores a programfor implementing functional blocks that performs various types ofprocessing as the program is read by CPU 75. Memory 73 stores as workmachine motion data, work machine motion information transmitted fromwheel loader 1.

Second processor 70 according to the embodiment generates a motion imageshowing a motion state during works by wheel loader 1 based on workmachine motion data stored in memory 73 and has the motion image shown.Work machine motion data will be described later. Though an example inwhich second processor 70 has a motion image shown in real time based onwork machine motion data, the motion image showing a motion state duringworks by wheel loader 1, is described in the present example, replayprocessing based on the work machine motion data stored in memory 73after the works by wheel loader 1 can also be performed. Replayprocessing includes both of processing of a still image at one certaintime point and processing of moving images that continuously change overtime.

CPU 75 implements various functional blocks based on a program stored inmemory 73. Specifically, CPU 75 includes a selector 80, a motion imagegenerator 82, a display controller 84, an event determination unit 86,and an event registration unit 88.

Selector 80 selects as time of replay, time in work machine motion datastored in memory 73 in replay processing. Motion image generator 82generates motion image data of wheel loader 1 based on work machinemotion data. Display controller 84 outputs to display 72, a work screenincluding a motion image based on the motion image data of wheel loader1 generated by motion image generator 82 and controls display 72 to showthe work screen. Event determination unit 86 determines whether or notan event has occurred based on the work machine motion informationstored in memory 73. Event registration unit 88 has memory 73 storetherein in association with the work machine motion information, eventinformation on the event determined as having occurred by eventdetermination unit 86. Second processor 70 corresponds to an exemplary“display system of a work machine” according to the present invention.Selector 80, motion image generator 82, event determination unit 86,event registration unit 88, display 72, and memory 73 correspond to anexemplary “selector”, an exemplary “motion image generator,” anexemplary “event determination unit,” an exemplary “event registrationunit,” an exemplary “display”, and an exemplary “storage” according tothe present invention, respectively.

FIG. 6 is a diagram illustrating a work machine table stored in memory73 according to the embodiment. Referring to FIG. 6, the work machinetable includes work machine motion data arranged in a time-seriesmanner.

By way of example, a plurality of pieces of work machine motion data arestored in the work machine table. Specifically, pieces of work machinemotion data corresponding to time points of works “12:01:01”,“12:01:05”, “12:01:10”, “12:02:00”, and “12:02:04” on a time line areshown. By way of example, time of the work corresponds to time ofreception by communication unit 74 of second processor 70, of data (workmachine motion information) transmitted from output unit 45 of firstprocessor 30. The time is not limited to time of reception of theinformation by communication unit 74 of second processor 70 but may betime of transmission by output unit 45 of first processor 30 or anotherreference time.

Work machine motion data includes work machine motion information andevent information associated with the work machine motion information.

Event information is information on an event that is occurring. Theevent information is set by event registration unit 88, althoughdescription will be provided later. Therefore, before setting by eventregistration unit 88, event information in work machine motion data isblank.

The work machine motion information includes vehicle information CN,operation information T, position information P, an operator ID, and avehicular body ID that are brought in correspondence with time of awork. Vehicle information CN is information on wheel loader 1.Specifically, vehicle information CN includes information on workimplement 3 and information on a vehicle including traveling unit 4except for work implement 3. Though an example in which vehicleinformation CN includes information on both of work implement 3 and thevehicle (the machine main body) is described in the present example, thevehicle information may include any one of them. Information on workimplement 3 includes work implement data relating to detection signalsfrom first angle detector 29, second angle detector 48, and firsthydraulic pressure detectors 28 a and 28 b and a work step. A state ofan attitude of work implement 3 can be sensed based on the workimplement data.

Information on the vehicle includes vehicle data relating to detectionsignals from temperature sensor 31, vehicle speed detection unit 27, andarticulation angle sensor 61. A state of traveling unit 4 can be sensedbased on the vehicle data. Operation information T includes workimplement operation information on work implement 3 and vehicleoperation information. The work implement operation information includeswork implement operation data relating to detection signals from boomoperation detection unit 52 b and bucket operation detection unit 54 b.A state of an operation onto work implement 3 can be sensed based on thework implement operation data. The vehicle operation informationincludes vehicle operation data relating to detection signals fromforward and rearward travel switching operation member 49 a, acceleratoroperation detection unit 51 b, shift change operation detection unit 53b, articulation operation detection unit 55 b, and brake operationdetection unit 58 b. A state of an operation onto the vehicle can besensed based on the vehicle operation data. Though an example in whichoperation information T includes both of work implement operationinformation and vehicle operation information is described in thepresent example, the operation information may include any one of them.

Position information P is information relating to a position of wheelloader 1. Specifically, position information P includes position datarelating to a detection signal from position detection sensor 64 andinclination data relating to a detection signal from IMU 66.

The operator ID is information for identification of an operator ofwheel loader 1. By way of example, the operator ID is stored in advancein a key used for start-up of the engine of the work machine by theoperator. First processor 30 obtains the operator ID from the key at thetime when the engine of the work machine is started up.

The vehicular body ID is information for identification of the vehicularbody of wheel loader 1. By way of example, the vehicular body ID isstored in advance in storage 30 j of first processor 30. Though anexample in which the vehicular body ID is stored in storage 30 j offirst processor 30 is described in the present example, the vehicularbody ID may be stored in advance in memory 73 of second processor 70without being limited as such.

[Event Registration Processing]

FIG. 7 is a flowchart illustrating event registration processing bysecond processor 70 according to the embodiment. Event registrationprocessing can be performed as background processing in parallel tomotion image generation processing for generating a motion image thatshows a motion state during works by wheel loader 1. Referring to FIG.7, event determination unit 86 obtains work machine motion data storedin the work machine table in memory 73 (step ST0).

Then, event determination unit 86 determines whether or not an event hasoccurred based on the obtained work machine motion data (step ST4).Event determination unit 86 determines whether or not the obtained workmachine motion data satisfies a prescribed event condition.

By way of example, regarding an event condition, when a temperature isequal to or higher than a prescribed temperature, an overheat event isdetermined as having occurred in the present example. For example,information on the vehicle included in vehicle information CN includesvehicle data relating to a detection signal from temperature sensor 31.Event determination unit 86 determines whether or not the temperature isequal to or higher than a prescribed temperature based on data obtainedfrom the detection signal from temperature sensor 31.

Regarding another event condition, in the present example, an event isdetermined as having occurred based on information on a work step. Forexample, vehicle information CN includes work implement data including awork step. Event determination unit 86 determines whether or not anevent has occurred based on the work implement data. When the workimplement data includes information on an excavation work step, anexcavation event is determined as having occurred.

Without being limited as such, various event conditions can be provided.A prescribed event may be determined as having occurred based on onepiece of data in the obtained work machine motion information or basedon combination of a plurality of pieces of data.

When event determination unit 86 determines in step ST4 that an eventhas occurred (YES in step ST4), it gives a registration instruction toevent registration unit 88.

Then, event registration unit 88 has event information registered inaccordance with the registration instruction from event determinationunit 86 (step ST6).

Then, event registration unit 88 determines whether or not checking ofall pieces of work machine motion data included in the work machinetable has ended (step ST8).

When event registration unit 88 determines in step ST8 that checking ofall pieces of work machine motion data has ended (YES in step ST8),event registration processing ends (end). When event registration unit88 determines in step ST8 that checking of all pieces of work machinemotion data has not ended (NO in step ST8), the process returns to stepST0. The work machine motion data in the work machine table for whichchecking has not ended is obtained and the processing above is repeated.

When an overheat event has occurred by way of example, eventregistration unit 88 has the event registered in the work machine motiondata, as event information associated with the work machine motioninformation. When an excavation event has occurred by way of example,event registration unit 88 has the event registered in the work machinemotion data, as event information associated with the work machinemotion information. This is also applicable to other pieces of workmachine motion data. Event information shown in FIG. 6 is thus set.

[Motion Image Generation Processing]

FIG. 8 is a diagram illustrating a detailed functional block of motionimage generator 82 according to the embodiment. Referring to FIG. 8,motion image generator 82 includes a motion state image generator 820, aposition state image generator 822, a vehicle state image generator 824,a management information image generator 826, and an event informationimage generator 828. Each functional block of motion image generator 82is implemented by a program stored in advance in memory 73.

Motion state image generator 820 generates motion state image data basedon the work machine motion data. Position state image generator 822generates position state image data based on the work machine motiondata. Vehicle state image generator 824 generates vehicle state imagedata based on the work machine motion data. Management information imagegenerator 826 generates management information image data based on thework machine motion data. Event information image generator 828generates event information image data based on the work machine motiondata.

FIG. 9 is a diagram illustrating a work screen 200 on display 72according to the embodiment. Referring to FIG. 9, work screen 200 isprovided with a plurality of screens where various types of informationon wheel loader 1 corresponding to time of a work “2018/1/1/12:02:00” ofwheel loader 1 are shown. The plurality of screens are insynchronization with the time of the work. In the present example,motion image generator 82 generates motion image data of wheel loader 1based on the work machine motion data corresponding to the time of thework stored in the work machine table. Time of replay may be informationonly on time or may include information on a date.

Display controller 84 controls display 72 to show work screen 200 basedon the motion image data generated by motion image generator 82. In thepresent example, motion image data includes motion state image data,position state image data, vehicle state image data, managementinformation image data, and event information image data.

Specifically, display controller 84 controls display 72 to show a motionscreen 210 where movement of wheel loader 1 is shown, based on themotion state image data. Display controller 84 controls display 72 toshow a position screen 230 where wheel loader 1 is shown, based on theposition state image data. Display controller 84 controls display 72 toshow a state screen 220 where information on a state of wheel loader 1is shown, based on the vehicle state image data. Display controller 84controls display 72 to show an event display screen 240 where a list ofevents that have occurred in wheel loader 1 is shown, based on the eventinformation image data. Display controller 84 controls display 72 toshow a management screen 260 where management information of wheelloader 1 is shown, based on the management information image data.

Display controller 84 may provide a command bar for giving variouscommands relating to replay processing to work screen 200. The commandbar includes a play button, a stop button, a pause button, a fastforward button, and a fast reverse button. As a manager operates thebutton, various types of processing relating to replay processing can beperformed. As the manager operates the play button, processing forcontinuously replaying motion images of wheel loader 1 (moving imagereplay processing) that change over time from a certain time of replayis performed. As the manager operates the pause button, processing forreplaying a motion image of wheel loader in accordance with a certaintime of replay (still image replay processing) is performed.

[Motion Screen 210]

Motion screen 210 is shown based on the motion state image data. Motionscreen 210 includes an inclined state image 212, an articulated stateimage 214, and a work state image 216.

Motion state image generator 820 generates first motion state image databased on 3D model geometrical data for generating a reference image of a3D model geometry of wheel loader 1 and work implement data included invehicle information CN. The 3D model geometrical data is stored inadvance in memory 73. The 3D model geometrical data includes data on amodel geometry of each of the work implement, the vehicular body, andthe wheel that composes a reference image of the 3D model geometry. Thefirst motion state image data may be image data representing aninclination or an articulation angle of the vehicle.

Display controller 84 has work state image 216 shown based on the firstmotion state image data, the work state image showing a state duringworks by wheel loader 1. Work state image 216 includes a work implementmodel image 217 showing a 3D model of wheel loader 1 and a road surfaceimage 218 that shows a road surface model on which wheel loader 1 runs.

Display controller 84 can allow expression of a state of running in workimplement model image 217 and road surface image 218 as being combined.Specifically, by way of example, forward movement of work implementmodel image 217 can be expressed by sliding road surface image 218 fromthe left to the right without change in position of work implement modelimage 217. Display controller 84 may express a degree of a state ofrunning in work implement model image 217 by adjusting a moving speed ofroad surface image 218. For example, display controller 84 may providesuch an expression that work implement model image 217 moves forward ata high speed by increasing a speed of sliding road surface image 218. Incontrast, display controller 84 may provide such an expression that workimplement model image 217 moves forward at a low speed by lowering aspeed of sliding road surface image 218. Display controller 84 mayadjust the speed of sliding road surface image 218 based on vehiclespeed data included in vehicle information CN. Display controller 84 haswork implement model image 217 shown based on first motion state imagedata. A state of works during works by wheel loader 1 can thus bevisualized. A field of view can also be changed by accepting setting ofa direction of the field of view of a virtual camera in visualization ofthe states during works. An example where work implement model image 217is arranged in a virtual space and picked up by a virtual camera from aprescribed image pick-up position is shown. The position of imagepick-up by the virtual camera can arbitrarily be set, and by adjustingthe position, representation resulting from pick-up of work implementmodel image 217 at an arbitrary angle can be provided.

Motion state image generator 820 generates second motion state imagedata based on side surface geometrical model data for generating areference image of a geometry of a side surface model of wheel loader 1and inclination data included in position information P. Displaycontroller 84 has inclined state image 212 shown based on the secondmotion state image data. Inclined state image 212 shows a state ofinclination of the vehicular body of wheel loader 1. In the presentexample, exemplary inclination by 25° is shown. Display controller 84has inclined state image 212 shown based on the second motion stateimage data. A state of inclination of the vehicular body during works bywheel loader 1 can thus be visualized.

Motion state image generator 820 generates third motion state image databased on top-view geometrical model data for generating a referenceimage of a geometry of a top view model of wheel loader 1 andarticulation angle data included in vehicle information CN. Displaycontroller 84 has an articulated state image 214 shown based on thethird motion state image data. Articulated state image 214 shows a stateof articulation of wheel loader 1. In the present example, a state ofright angling by 15° is shown. Display controller 84 has articulatedstate image 214 shown based on the third motion state image data. Anarticulated state during works by wheel loader 1 can thus be visualized.

[State Screen 220]

State screen 220 is shown based on vehicle state image data. Statescreen 220 includes a time bar 224 that indicates time of a work duringworks, an operation state image 221, a vehicle state image 222, and awork step state image 223. Time bar 224 indicates time of a work duringworks. Time bar 224 in the present example is provided as being movableto a position corresponding to any time of work.

Vehicle state image generator 824 generates first vehicle state imagedata based on operation information T in work machine motion data duringa period until a time point a prescribed period before the current timeof work. Display controller 84 has operation state image 221 shown basedon the first vehicle state image data, the operation state image showinga state over a prescribed period of the operation member operated by anoperator. Operation state image 221 includes a designated operationstate image 221A corresponding to current time of work and an operationtransition state image 221B showing a state of transition of operationsduring the period until the time point the prescribed period before thecurrent time of work. Designated operation state image 221A shows astate of accelerator operation apparatus 51 (an accelerator pedal), boomoperation apparatus 52 (a boom lever), bucket operation apparatus 54 (abucket lever), and brake operation apparatus 58 (a brake). In thepresent example, 95% for the accelerator pedal, 25% for the boom lever,14% for the bucket lever, and a brake OFF state are shown. Operationtransition state image 221B shows a state of transition of operationsonto accelerator operation apparatus 51 (accelerator pedal), boomoperation apparatus 52 (boom lever), bucket operation apparatus 54(bucket lever), and brake operation apparatus 58 (brake) over theprescribed period (26 s in the present example).

An operation state is shown in a grayscale for accelerator operationapparatus 51 (accelerator pedal), boom operation apparatus 52 (boomlever), and bucket operation apparatus 54 (bucket lever). Specifically,a value of a ratio of operations onto the operation member is larger,the color is denser (blacker), and as a numeric value is smaller, thecolor is lighter (whiter). Though an example where an operation state isexpressed with the grayscale is described in the present example, theoperation state may be shown with a heat map. For example, by changing acolor at the time when boom operation apparatus 52 (boom lever) isoperated to perform a raising motion or a lowering motion, whichoperation is being performed can visually intuitively be known. This isalso applicable to bucket operation apparatus 54 (bucket lever). A valueof an amount of operations onto an operation member may be shown in agraph. For brake operation apparatus 58 (brake), a brake ON state or abrake OFF state is shown. What kind of operation has been performed onan operation member for a prescribed period can thus readily be known.

Vehicle state image generator 824 generates second vehicle state imagedata based on vehicle information CN in work machine motion data duringa period until a time point a prescribed period before the current timeof work. Display controller 84 has vehicle state image 222 and work stepstate image 223 shown based on the second vehicle state image data, thevehicle state image and the work step state image showing a state ofwheel loader 1 over a prescribed period. Vehicle state image 222includes a designated vehicle state image 222A corresponding to currenttime of work and a vehicle transition state image 222B showing a stateof transition of the vehicle during the period until the time point theprescribed period before the current time of work. In the presentexample, designated vehicle state image 222A shows an example where avehicle speed of wheel loader 1 is at 15 km/h.

Vehicle transition state image 222B shows a state of transition of thevehicle speed of wheel loader 1 over the prescribed period (26 s in thepresent example). How the vehicle speed of wheel loader 1 has variedover the prescribed period can thus readily be known.

Work step state image 223 includes a designated work step state image223A that shows a work step corresponding to current time of work and awork step transition state image 223B that shows a state of transitionof work steps during the period until the time point the prescribedperiod before the current time of work. Designated work step state image223A shows excavation in the present example. Work step transition stateimage 223B shows an example in which the work step is varied to unloadedforward travel, excavation, and loaded forward travel during theprescribed period. How the work steps of wheel loader 1 have variedduring the prescribed period can thus readily be known.

Display controller 84 controls operation transition state image 221B,vehicle transition state image 222B, and work step transition stateimage 223B to move from the right to the left in accordance with time ofwork corresponding to time bar 224 by way of example. Time bar 224 canbe moved to a position of any time of work during the prescribed periodbased on input through input unit 71.

[Position Screen 230]

Position screen 230 is shown based on the position state image data.Position screen 230 includes a work position image 232 and a movementtrack image 234.

Position state image generator 822 generates position state image databased on map data that shows a work map and position data included inposition information P. Display controller 84 has work position image232 and movement track image 234 shown based on the position state imagedata, the work position image showing a position of works by wheelloader 1 on the work map, the movement track image showing a track ofmovement of wheel loader 1. Work position image 232 is provided as beingmovable in accordance with movement track image 234 that shows a trackof movement of wheel loader 1. Display controller 84 has work positionimage 232 and movement track image 234 shown on the work map based onposition state image data. A state of movement during works by wheelloader 1 can thus be visualized. Movement track image 234 includes aspeed change region 233. Speed change region 233 refers to a regionwhere a speed of movement of wheel loader 1 has changed. By providingspeed change region 233 in movement track image 234, change in speed ofwheel loader 1 can visually be determined. Though one speed changeregion 233 is shown in the present example, a plurality of regions maybe provided without particularly being limited as such. Change in speedis shown by change in hatching pattern by way of example. Without beinglimited to such representation, change in speed of wheel loader 1 mayvisually be determined based on a color or another highlightedrepresentation. Work position image 234 can be changed to any positionof works along movement track image 234 based on input through inputunit 71.

[Event Display Screen 240]

Event display screen 240 is shown based on event information image data.Event display screen 240 includes an event list. Event information imagegenerator 828 generates event information image data based on eventinformation in work machine motion data. Display controller 84 has theevent list shown based on the event information image data. In thepresent example, overheat and a work step are shown as events. Worksteps are further subcategorized and steps such as excavation andunloaded forward travel are listed. Time of each event is shown in atree format.

In the present example, selection from among events shown in the treeformat can be made based on an item indicating time of occurrence ofthat event. For example, by designating an item “2018/1/1/12:20:00” thatindicates time of overheat, work machine motion data in the work machinetable brought in correspondence with that time is selected. Then, motionimage data of wheel loader 1 is generated based on the selected workmachine motion data. A state of occurrence of an event during works bywheel loader 1 can thus readily be visualized.

[Management Screen 260]

Management screen 260 is shown based on management information imagedata. Management screen 260 includes a time image 202 that shows time ofa work, a vehicular body ID image 204, and an operator ID image 206.Management information image generator 826 generates first managementinformation image data based on time information in work machine motiondata. Display controller 84 has time image 202 corresponding to time ofa work shown based on the first management information image data. Inthe present example, “2018/1/1:12:02:00” is shown as time image 202.

Management information image generator 826 generates second managementinformation image data based on a vehicular body ID in the work machinemotion data. Display controller 84 has vehicular body ID image 204 shownbased on the second management information image data. In the presentexample, “X” is shown as vehicular body ID image 204.

Management information image generator 826 generates third managementinformation image data based on the operator ID in the work machinemotion data. Display controller 84 has operator ID image 206 shown basedon the third management information image data. In the present example,“A” is shown as the operator ID image. Display controller 84 has time,the vehicular body ID, and the operator ID shown based on the first tothird management information image data. Management information duringworks by wheel loader 1 can thus readily be visualized.

[Replay Position Selection Processing]

In the embodiment, a manager can select a position to replay a motionimage of wheel loader 1. FIG. 10 is a flowchart illustrating replayposition selection processing by second processor 70 according to theembodiment. Referring to FIG. 10, selector 80 determines whether or notit has accepted input from input unit 71 (step S2). When selector 80 hasnot accepted input from input unit 71, it maintains a state in step S2.

When selector 80 determines that it has accepted input from input unit71 (YES in step S2), it determines whether or not the input is operationinput onto time bar 224 (step S4). When selector 80 determines that theinput is not the operation input onto time bar 224 (NO in step S4), theprocess proceeds to step S20.

When selector 80 determines that the input is operation input onto timebar 224 (YES in step S4), it accepts a command to select a position oftime bar 224 (step S6). For example, a position of end of an operationthrough input unit 71 is accepted as the command to select a position oftime bar 224.

Then, selector 80 selects time of work corresponding to the position oftime bar 224 as time of replay (step S9). Then, motion image generator82 generates motion image data of wheel loader 1 based on work machinemotion data, in accordance with the time of replay selected by selector80 (step S10).

Then, display controller 84 performs replay processing based on themotion image data of wheel loader 1 generated by motion image generator82 (step S12). Specifically, as described with reference to FIG. 8,display controller 84 outputs a work screen including a motion image todisplay 72 and controls the display to show the work screen. Byselecting a position of time bar 224, the manager can perform processingfor replaying the motion image at the replay position corresponding toan arbitrary position of time bar 224.

Then, display controller 84 determines whether or not replay processinghas ended (step S14). When display controller 84 determines that replayhas not ended (NO in step S14), the process returns to step S2 and theprocessing above is repeated.

When display controller 84 determines that replay processing has ended(YES in step S14), the process ends. When selector 80 determines in stepS20 that an operation onto time bar 224 has not been performed, itdetermines whether or not an operation onto work position image 232 hasbeen performed. When selector 80 determines that an operation onto workposition image 232 has not been performed (NO in step S20), the processproceeds to step S24.

When selector 80 determines that an operation onto work position image232 has been performed (YES in step S20), it accepts a command to selecta work position (step S22). For example, a position of end of anoperation using input unit 71 is accepted as a command to select aposition of work position image 232. Work position image 232 is providedas being movable as following a track in movement track image 234.

Then, selector 80 selects time of work corresponding to the position ofwork position image 232 as time of replay (step S9). Since subsequentprocessing is similar to the above, detailed description thereof willnot be repeated.

The manager can perform processing for replaying a motion image at areplay position corresponding to an arbitrary position of work positionimage 232 by selecting the position of work position image 232. Whenselector 80 determines that an operation onto work position image 232has not been performed, it determines whether or not an operation ontoan event list has been performed.

When selector 80 determines that an operation onto the event list hasnot been performed (NO in step S24), the process returns to step S2.When selector 80 determines that an operation onto the event list hasbeen performed (YES in step S24), it accepts a command to select eventinformation (step S26). For example, a command to select eventinformation designated by an operation onto input unit 71 is accepted.For example, input to select time corresponding to excavation oroverheat is accepted.

Then, selector 80 selects time of acceptance of selection input as timeof replay (step S9). Since subsequent processing is similar to theabove, detailed description thereof will not be repeated. The managercan perform processing for replaying a motion image at a replay positioncorresponding to an arbitrary position of an event by performing anoperation onto the event list.

[Form of Use]

Work screen 200 is shown as a result of motion image processing bysecond processor 70 according to the embodiment. Work screen 200 showsmotion screen 210, state screen 220, and position screen 230 generatedbased on work machine motion data corresponding to time of work duringworks by wheel loader 1 by way of example.

In motion screen 210, by way of example, work state image 216 showing astate of works by wheel loader 1 is shown. In state screen 220, by wayof example, operation state image 221 is shown. In position screen 230,work position image 232 showing a position of works by wheel loader 1 ona work map is shown. The manager can readily know on the screens, when,where, and what kind of work an operator of wheel loader 1 has done.

Since the manager can readily recognize not only a motion state of wheelloader 1 but also an operation state, for example, the manager can makeeffective use of the same in training driving by an operator. Themanager can provide appropriate training relating to a state of anoperation by an operator that corresponds to a motion state of the workmachine, by checking, for example, operation state image 221 on statescreen 220 together with motion screen 210. The manager can provideappropriate training relating to a state of an operation by the operatorthat corresponds to a position state of the work machine, by checking,for example, operation state image 221 on state screen 220 together withposition screen 230.

Since the manager can arbitrarily select a position to replay a motionimage of wheel loader 1, for example, the manager can make effective useof the same in training driving by an operator. By checking the eventlist, it can also be made use of for trouble shooting or investigationof complaints.

Other Embodiments

Though an example in which the work machine table is stored in memory 73of second processor 70 is described in the embodiment, it may be stored,for example, in storage 30 j of first processor 30 without particularlybeing limited as such. Motion image processing may be performed based onthe work machine table stored in storage 30 j of first processor 30.

Though a configuration in which various functional blocks areimplemented by CPU 75 of second processor 70 is described in theembodiment, some or all of the functional blocks may be implemented byfirst processor 30 without being limited as such.

Though an example in which work screen 200 is shown on display 72 ofsecond processor 70 is described in the embodiment, work screen 200 maybe shown on display 40 of wheel loader 1. Display 40 and first processor30 of wheel loader 1 may be integrated into one device. Without beinglimited to wheel loader 1, for example, work screen 200 may be shown ona display of a portable terminal provided to communicate with secondprocessor 70.

An example in which a plurality of screens in synchronization with timeof a work are shown on work screen 200 is described in the embodiment.Specifically, though an example in which work screen 200 includes motionscreen 210, state screen 220, position screen 230, event display screen240, and management screen 260 as the plurality of screens is described,all of these screens do not particularly have to be shown, and forexample, two or more screens may be shown. For example, motion screen210 and state screen 220 may be shown on work screen 200. Combinationwith another screen can also naturally be made.

[Functions and Effects]

Functions and effects of the embodiment will now be described. As shownin FIG. 5, in the display system of the work machine in the embodiment,second processor 70 is provided with communication unit 74 that receiveswork machine motion information transmitted from wheel loader 1, motionimage generator 82 that generates a motion image of wheel loader 1 andan operation image corresponding to the motion image based on the workmachine motion information received at communication unit 74, anddisplay 72 that shows the motion image and the operation image. As shownin FIG. 6, the work machine motion information includes motioninformation and operation information including vehicle information andposition information. The motion image of wheel loader 1 and theoperation image corresponding to the motion image are generated based onthe motion information and the operation information, and the motionimage and the operation image are shown on display 72. Therefore, theoperation state of wheel loader 1 together with the motion state ofwheel loader 1 can also readily be recognized.

The motion information and the operation information may be obtained byfirst processor 30 within wheel loader 1 or second processor 70. Sincethe motion information and the operation information can be arrangedalso in the second processor which is another external device withoutbeing limited to the first processor within wheel loader 1, a degree offreedom of the display system can be improved.

Motion image generator 82 is provided in first processor 30 within wheelloader 1 or in second processor 70. Since the motion image generator canbe arranged also in another external device without being limited towheel loader 1, a degree of freedom of the display system of the workmachine can be improved.

Display 72 is provided in first processor 30 within wheel loader 1 or insecond processor 70. Since the display can be arranged also in anotherexternal device without being limited to wheel loader 1, a degree offreedom of the display system of the work machine can be improved.

The display system of the work machine in the embodiment is providedwith memory 73 that stores the work machine motion information receivedat communication unit 74. As shown in FIG. 6, motion image generator 82generates a motion image of the work machine based on the work machinemotion information stored in memory 73 and generates an operation imagecorresponding to the motion image of the work machine based on the workmachine motion information stored in memory 73. By storing images inmemory 73, the motion image showing the motion state of wheel loader 1and the operation image showing the operation state can be subjected toreplay processing, and they can be made effective use of in trainingdriving by an operator.

Wheel loader 1 includes work implement 3 and traveling unit 4. The workmachine motion information shown in FIG. 6 includes vehicle informationCN, and vehicle information CN includes motion information of at leastone of work implement 3 and traveling unit 4. A motion state and anoperation state corresponding thereto, of at least one of work implement3 and traveling unit 4 of wheel loader 1 can readily be recognized.

The work machine motion information shown in FIG. 6 further includesidentification information for identification of an operator or avehicular body. When a plurality of operators or a plurality ofvehicular bodies are provided, they can readily be distinguished fromone another based on the identification information.

Wheel loader 1 is further provided with selector 80 that selects as timeof replay, time in the work machine motion information. Selector 80selects time to replay the work machine motion information in accordancewith an input command provided through input unit 71 onto work screen200 shown in FIG. 9. A motion state of wheel loader 1 can be replayedand checked in a simplified manner by using an input interface of workscreen 200.

Wheel loader 1 includes boom 14 and bucket 6. The work machine motioninformation shown in FIG. 6 includes vehicle information CN, and vehicleinformation CN includes a detection signal from second angle detectionunit 48 that detects a tilt angle of bucket 6 with respect to boom 14. Astate of an angle of bucket 6 with respect to boom 14 of wheel loader 1can readily be recognized.

The work machine motion information shown in FIG. 6 includes operationinformation T, and operation information T includes a detection signalfrom bucket operation detection unit 54 b. A state of an operation ontobucket 6 of wheel loader 1 can readily be recognized.

The display system of the work machine in the embodiment is exemplifiedby a display system of wheel loader 1, and an operation state togetherwith a motion state of wheel loader 1 can also readily be recognized.

A method of controlling a display system of a work machine in theembodiment includes obtaining at least motion information and operationinformation of the work machine, generating a motion image of the workmachine and an operation image corresponding to the motion image of thework machine based on the obtained motion information and operationinformation, and showing the motion image and the operation image of thework machine. The motion image of wheel loader 1 and the operation imagecorresponding to the motion image are generated based on the motioninformation and the operation information, and the motion image and theoperation image are shown on display 72. Therefore, the operation statetogether with the motion state of wheel loader 1 can also readily berecognized.

Though a wheel loader is described as the work machine by way ofexample, a work machine such as a hydraulic excavator, a dump truck, ora crawler dozer is also applicable.

Though an embodiment of the present invention has been described above,it should be understood that the embodiment disclosed herein isillustrative and non-restrictive in every respect. The scope of thepresent invention is defined by the terms of the claims and is intendedto include any modifications within the scope and meaning equivalent tothe terms of the claims.

REFERENCE SIGNS LIST

1 wheel loader; 2 vehicular body frame; 3 work implement; 4 travelingunit; 5 cab; 6 bucket; 6 a cutting edge; 10 boom pin; 13 steeringcylinder; 14 boom; 15 tilt rod; 16 boom cylinder; 17 bucket pin; 18 bellcrank; 18 a support pin; 19 tilt cylinder; 20 engine; 21 input shaft; 22motive power extraction unit; 23 motive power transmission mechanism; 23a output shaft; 24 cylinder driving unit; 25 work implement pump; 26control valve; 27 vehicle speed detection unit; 28 a first hydraulicpressure detector; 29 first angle detector; 30 first processor; 30 jstorage; 31 temperature sensor; 40, 72 display; 45 output unit; 48second angle detector; 49 forward and rearward travel switchingapparatus; 49 a forward and rearward travel switching operation member;49 b forward and rearward travel switching detection sensor; 51accelerator operation apparatus; 51 a accelerator operation member; 51 baccelerator operation detection unit; 52 boom operation apparatus; 52 aboom operation member; 52 b boom operation detection unit; 53 shiftchange operation apparatus; 53 a shift change operation member; 53 bshift change operation detection unit; 54 bucket operation apparatus; 54a bucket operation member; 54 b bucket operation detection unit; 55articulation operation apparatus; 55 a articulation operation member; 55b articulation operation detection unit; 58 brake operation apparatus;58 a brake operation member; 58 b brake operation detection unit; 60pivot mechanism; 61 articulation angle sensor; 64 position detectionsensor; 70 second processor; 71 input unit; 73 memory; 74 communicationunit; 80 selector; 82 motion image generator; 84 display controller; 86event generator; 88 event registration unit

1. A display system of a work machine comprising: an obtaining unit thatobtains motion information and operation information of the workmachine; a motion image generator that generates a motion image of thework machine and an operation image corresponding to the motion image ofthe work machine based on the motion information and the operationinformation obtained by the obtaining unit; and a display that shows themotion image and the operation image of the work machine.
 2. The displaysystem of the work machine according to claim 1, wherein the obtainingunit is provided in any one of the work machine and an external deviceindependent of the work machine.
 3. The display system of the workmachine according to claim 1, wherein the motion image generator isprovided in any one of the work machine and an external deviceindependent of the work machine.
 4. The display system of the workmachine according to claim 1, wherein the display is provided in any oneof the work machine and an external device independent of the workmachine.
 5. The display system of the work machine according to claim 1,further comprising a storage that stores work machine motion informationincluding the motion information and the operation information of thework machine obtained by the obtaining unit, wherein the motion imagegenerator generates the motion image of the work machine based on thework machine motion information stored in the storage, and generates theoperation image corresponding to the motion image of the work machinebased on the work machine motion information stored in the storage. 6.The display system of the work machine according to claim 1, wherein thework machine includes a work implement and a machine main body, and themotion information of the work machine includes motion information of atleast one of the work implement and the machine main body.
 7. Thedisplay system of the work machine according to claim 5, wherein thework machine motion information further includes identificationinformation.
 8. The display system of the work machine according toclaim 5, further comprising a selector that selects as time of replay,time in the work machine motion information.
 9. The display system ofthe work machine according to claim 1, wherein the work machine includesa boom and a bucket, and the motion information of the work machineincludes an angle of the bucket with respect to the boom.
 10. Thedisplay system of the work machine according to claim 9, wherein theoperation information of the work machine includes a detection signalrepresenting a command to operate the bucket.
 11. The display system ofthe work machine according to claim 1, wherein the work machine is awheel loader.
 12. A method of controlling a display system of a workmachine comprising: obtaining motion information and operationinformation of the work machine; generating a motion image of the workmachine and an operation image corresponding to the motion image of thework machine based on the obtained motion information and operationinformation; and showing the motion image and the operation image of thework machine.
 13. The display system of the work machine according toclaim 10, wherein the operation information includes vehicle operationinformation of the work machine.